The Internet-based Teleoperation: Motion and Force Predictions Using the Particle Filter Method

In this paper, we present motion and force predictions in Internet-based teleoperation systems using the particle filter method. The particle filter, also known as the sequential Monte Carlo (SMC) method, is a probabilistic prediction or estimation technique within a sequential Bayesian framework: Data at a current time step are predicted or estimated by recursively generating probability distribution based on previous observations and input states. In this paper, we first formulate the particle filter method using a prediction-based approach. Motion and force data flows, which may be impaired by the Internet delay, are formulated within a sequential Bayesian framework. The true motion and force data are then predicted by employing the prediction-based particle filter method using the impaired observations and previous input states. We performed experiments using a haptic device that interacts with a mechanics-based virtual 3D graphical environment. The haptic device is used as a master controller that provides positioning inputs to a 4-degree of freedom (4-DoF) virtual robotic manipulator while receiving feedback force through interactions with the virtual environment. We simulate the Internet delay with variations typically observed in a user datagram protocol (UDP) transmission between the master controller and the virtual teleoperated robot. In this experimental scenario, the particle filter method is implemented for both motion and force data that experience the Internet delay. The proposed method is compared with the conventional Kalman filter. Experimental results indicate that in nonlinear and non-Gaussian environments the prediction-based particle filter has distinct advantage over other methods. INTRODUCTION An Internet-based teleoperation system is an interactive application where though a master device, a human operator transmits motion data while simultaneously receiving reflecting force data from a slave robot controller. Unlike other Internet applications that mainly focus on the reliable data transmission, interactive applications are highly delay-sensitive. The Internet delay, which is unknown and varies over time according to network conditions, may cause instability of an overall teleoperation system. Furthermore, the transmitted motion and force data are often impaired by significant delay and delay jitter during the Internet transmission [1]. Various approaches have been suggested in order to solve the time delay issue of Internet-based teleoperation systems. In the area of control systems, the wave variables transformation and its extensions have focused on the stability of overall teleoperation systems in the presence of constant delay [2], [3], 1 Copyright © 2010 by ASME [4]. In the area of Internet transport protocols, several proposals have been suggested based on modifications to transport control protocol (TCP) and user datagram protocol (UDP) to enable faster transmissions of data packets [5], [6]. In the area of signal processing, prediction-based methods that perform motion and force predictions have been proposed [7], [8]. The Kalman filter method, which provides a recursive solution to the linear prediction and estimation, was proposed as a prediction-based approach [9]. These methods have been used to compensate for the transmitted motion and force data that are impaired by variations of the Internet delay. Motion and force data are often difficult to predict in systems with nonlinear and non-Gaussian characteristics. For example, fine hand motion commands from a master controller may be highly nonlinear and the traditional Kalman filter may fail to provide their accurate prediction. Force data may be even more difficult to predict since the data need to be sent at relatively high frequencies to guarantee realistic force without discontinuity and to avoid closed-loop instability of an overall teleoperation system. Furthermore, the motion and force data may contain a non-Gaussian noise such as an impulse noise during the transmission, which leads to further challenges in prediction. The particle filter method, also known as the bootstrap filter or the Condensation, is a sequential Monte Carlo (SMC) method that provides a sub-optimal solution in recursive Bayesian approaches [10], [11]. Due to its robust prediction and estimation performance in nonlinear and non-Gaussian environments, the particle filter method has been widely applied in the areas of communications, image and speech signal processing, control systems, and robotics [12], [13]. Since the particle filter method can be applied to any signal using a discrete time state-space formulation, it has been applied to nonlinear motion and force data flows in an Internet-based teleoperation system [14]. In this paper, we employ the particle filter method to predict motion and force data that may be nonlinear and nonGaussian in addition to being subject to the Internet delay. We first introduce the prediction-based particle filter method applied to motion and force data flows using a discrete time state-space formulation. We describe an experimental study based on the implemented particle filter method [14]. In this paper, the proposed method is verified in both nonlinear and non-Gaussian environments. We also present the comparison of prediction performance between the proposed particle filter and the conventional Kalman filter methods. The stability issue of an overall teleoperation system in the presence of the Internet delay is discussed. MOTION AND FORCE PREDICTIONS IN INTERNETBASED TELEOPERATION SYSTEMS Motion data generated from a master controller are transmitted to a slave controller through the Internet. Based on the design of the slave controller, reflected force data are generated by any contact with an object or surrounding environment and they are fed into the master controller through the Internet. A simple illustration of motion and force data flows in an Internet-based teleoperation system is shown in Figure 1. The motion and force data are represented in discrete time statespace formulations, which are configured in a recursive Bayesian framework. Since the motion and force data experience variations of the Internet delay, the true data may be impaired and stability of an overall teleoperation system may not be maintained. In order to compensate for such Internet delay, we employ the prediction-based particle filter method for the motion and force data flows.